Skip to main content

Geoffrey Hinton: Deep learning with multiplicative interactions

Movies Preview

movies
Geoffrey Hinton: Deep learning with multiplicative interactions


Published March 22, 2010


Talk by Geoffrey Hinton, University of Toronto and Canadian Institute for Advanced Research. Given to the Redwood Center for Theoretical Neuroscience on March 22, 2010.

Abstract.
Deep networks can be learned efficiently from unlabeled data. The layers of representation are learned one at a time using a simple learning module that has only one layer of latent variables. The values of the latent variables of one module form the data for training the next module. The most commonly used modules are Restricted Boltzmann Machines or autoencoders with a sparsity penalty on the hidden activities. Although deep networks have been quite successful for tasks such as object recognition, information retrieval, and modeling motion capture data, the simple learning modules do not have multiplicative interactions which are very useful for some types of data.

The talk will show how a third-order energy function can be factorized to yield a simple learning module that retains advantageous properties of a Restricted Boltzmann Machine such as very simple exact inference and a very simple learning rule based on pair-wise statistics. The new module has a structure that is very similar to the simple cell/complex cell hierarchy that is found in visual cortex. The multiplicative interactions are useful for modeling images, image transformations, and different styles of human walking.


Producer Redwood Center for Theoretical Neuroscience
Audio/Visual sound, color

comment
Reviews

There are no reviews yet. Be the first one to write a review.
SIMILAR ITEMS (based on metadata)
eye
Title
Date Archived
Creator
Community Video
by Redwood Center for Theoretical Neuroscience
movies
eye 279
favorite 0
comment 0
Community Video
by Redwood Center for Theoretical Neuroscience
movies
eye 300
favorite 0
comment 0
Community Video
by Redwood Center for Theoretical Neuroscience
movies
eye 439
favorite 0
comment 0
Community Video
by Redwood Center for Theoretical Neuroscience
movies
eye 446
favorite 0
comment 1
favoritefavoritefavoritefavoritefavorite ( 1 reviews )
Community Video
by Redwood Center for Theoretical Neuroscience
movies
eye 366
favorite 0
comment 0
Community Video
by Redwood Center for Theoretical Neuroscience
movies
eye 1,017
favorite 0
comment 0
Community Video
movies
eye 1,073
favorite 0
comment 0
Community Video
by Redwood Center for Theoretical Neuroscience
movies
eye 236
favorite 0
comment 0
Community Video
movies
eye 730
favorite 0
comment 0
Community Video
movies
eye 102
favorite 0
comment 0